Super-Resolution Enhancement of Digital Video

When designing a system for image acquisition, there is generally a desire for high spatial resolution and a wide field of view. To satisfy these competing requirements, a camera system typically must employ small f-number optics, which produces an image with very high spatial-frequency band-width at the focal plane. To avoid aliasing caused by under-sampling, the corresponding focal plane array (FPA) must be sufficiently dense. However, cost and fabrication complexities may make this impractical. More fundamentally, smaller detectors capture fewer photons, which can lead to potentially severe noise levels in the acquired imagery. Considering these factors, one may choose to accept a certain level of undersampling or to sacrifice some optical resolution and/or field of view. In super-resolution (SR) image reconstruction, postpro-cessing is used to obtain images with resolutions that go beyond the conventional limits of the uncompensated imaging system. In some systems, the primary limiting factor is the optical resolution of the image in the focal plane as defined by the cutoff frequency of the optics. We use the term " optical SR " to refer to SR methods that aim to create an image with valid spatial-frequency content that goes beyond this cutoff frequency. Such techniques typically must rely on extensive a priori information. In other image acquisition systems, the limiting factor may be the density of the FPA, subsequent postprocessing requirements, or transmission bit rate constraints that require data compression. We refer to the process of overcoming the limitations of the FPA in order to obtain the full resolution afforded by the selected optics as " detector SR. " Some methods may seek to perform both optical and detector SR. Detector SR algorithms generally process a set of low-resolution aliased frames from a video sequence to produce a high-resolution frame. When subpixel relative motion is present between the objects in the scene and the detector array , a unique set of scene samples is acquired for each frame. This provides the mechanism for effectively increasing the spatial sampling rate of the imaging system without reducing the physical size of the detectors. With the proliferation of digital imaging and video, SR has become a rapidly growing field. Recent advances in SR include innovative algorithms, generalized methods, real-time implementations, and novel applications. The purpose of this special issue is to present leading research and development in the area of super-resolution image reconstruction as applied to digital video. Specific topics …